This invention relates to video surveillance systems, specifically to a system that will automatically track a person as he/she moves around.
Many retail stores use video surveillance of customers to prevent shoplifting. Two types of cameras are used for this purpose: fixed cameras, and. pan/tilt/zoom cameras (PTZ""s). The fixed cameras are stationary, and are used to constantly monitor the same location. The PTZ cameras can be rotated around, and are used to zoom in on a particular location or person. Stores generally have a mixture of both types of cameras mounted on the ceiling throughout the store, with the resulting footage from these cameras being recorded for potential use as future evidence.
The field of view of a fixed camera is constant, so it is usually set up to cover a broad area, or one of particular interest (e.g. a jewelry counter or cash register). If it is covering a broad area, such as the whole music department, a single person will only comprise a small part of the entire image. This can make it difficult to determine from the image exactly what a person is doing. It might be impossible to distinguish a shoplifting incident, such as a person putting a CD into a pocket, from a normal gesture, such as someone""s sticking his hand into his jacket pocket.
The limitations of fixed cameras give rise to the use of PTZ cameras. If the camera is pointed and zoomed in appropriately, a person can be observed in more detail, so that innocent actions can be distinguished from theft. For this reason, it is common practice for security personnel to monitor customers by following them with PTZ""s as they move around the store.
In the prior art, the PTZ""s are controlled with joysticks or a mouse and computer interface, with the guard being required to manually follow a customer with the camera as they move about. This manual tracking process has the following disadvantages:
1) the guard needs to devote his/her full attention to the tracking process;
2) it is impossible for a single guard to track more than one subject simultaneously;
3) limitations in human manual dexterity make resulting video coverage jerky and incomplete;
4) limited human attention spans can result in a guard""s losing interest in a subject, and ceasing to track him, before the subject has left the store;
5) the process of shifting from one PTZ camera to a different one, as the subject moves from one region in the store to another, is a difficult process to do manually and usually results in a gap in coverage; and
6) if the guard is involuntarily distracted, e.g. with a phone call, there will be a gap in coverage.
The area of tracking moving objects with cameras has a long history, with origins in military applications. U.S. Pat. No. 3,953,669 to Saccomani (1976) discloses the selection of a target with a light pen, with a relatively simple procedure for producing an error signal allowing the window to be moved to keep the target centered. U.S. Pat. No. 3,988,534 to Sacks (1976) discloses another system with an automatically generated xe2x80x9ctracking gatexe2x80x9d target selection, with another procedure for producing an error signal that controls a servo allowing the camera to keep the target centered. Both of these systems are for use with single cameras only, and are limited to extremely high contrast situations where the target is extremely visually distinct from the background, due to the simplicity of the tracking algorithms.
More recently, U.S. Pat. No. 5,434,617 to Bianchi (1995) combines a fixed camera and a PTZ camera to track an individual, with the tracking algorithm based on pixel differencing techniques calculated on the fixed camera view, with a resulting motion control signal used to point the PTZ camera. This system requires that a fixed camera always be used in conjunction with the PTZ, and is once again limited to a single PTZ camera""s field of view. The system does have a slightly more robust tracking algorithm than those of Saccomani and Sacks, but will still not hold up against a complex background.
Other recent patents (e.g. U.S. Pat. Nos. 5,552,823 (1996), 5,706,362 (1998), 5,714,999 (1998), and 5,754,225 (1998)) are similar in that they disclose systems implementing algorithms for tracking individuals with cameras, with varying means of designating targets and varying complexity/robustness levels of the algorithms. All of these prior art systems are limited to single camera usage, and do not discuss the issues arising from multiple camera usage with hand-offs and self-configuration, that are addressed by the current invention.
Some computer automated tracking systems for outdoor vehicle tracking (e.g. in large facility yards) are currently offered for sale commercially, such as the e.clips(trademark) system from Cortex, Inc. This system is limited to controlling a single PTZ camera as well, however, and does not perform the xe2x80x9chand-offxe2x80x9d or auto-configuration procedures necessary for use with multiple cameras.
There are no systems that are currently offered commercially, and none known in the art, for automatically tracking individuals using multiple cameras as they move throughout stores, as disclosed by this invention.
Thus, the present invention, with computer automated tracking of individuals, has several advantages over the current manual tracking methods. Specifically, it:
1) allows the guard to concentrate on other activities, as necessary, after the initial designation of the subject to be tracked;
2) allows more than one subject to be tracked simultaneously, subject to the positions of the subjects and the number of available cameras;
3) results in smoother and more complete video coverage;
4) tracks a subject continuously, for as long as necessary, until the subject leaves the store;
5) allows seamless shifting from one PTZ camera to a different one, as the subject moves from one region in the store to another;
6) does not matter if the guard becomes involuntarily distracted temporarily, since the system will continue to track automatically.
Further objects and advantages include allowing integration of the tracking function with other computer automation, so that, for instance, a database of known previous shoplifters may be maintained, and enabling selection of potential tracking targets automatically through the use of incorporated face recognition software. Still further objects and advantages will become apparent from a consideration of the following detailed description, the accompanying drawings, and the claims.